AutoTransition: Learning to Recommend Video Transition Effects

AbstractVideo transition effects are widely used in video editing to connect shots for creating cohesive and visually appealing videos. However, it is challenging non-professionals choose best transitions due the lack of cinematographic knowledge design skills. In this paper, we present premier work on performing automatic recommendation (VTR): given a sequence raw companion audio, recommend each pair neighboring shots. To solve task, collect large-scale dataset using publicly available templates softwares. Then formulate VTR as multi-modal retrieval problem from vision/audio propose novel matching framework which consists two parts. First learn embedding through classification task. model correspondence inputs transitions. Specifically, proposed employs transformer fuse vision audio information, well capture context cues sequential outputs. Through both quantitative qualitative experiments, clearly demonstrate effectiveness our method. Notably, comprehensive user study, method receives comparable scores compared with professional editors while improving efficiency by 300 \(\boldsymbol{\times }\). We hope serves inspire other researchers new The codes public at https://github.com/acherstyx/AutoTransition.KeywordsVideo recommendationMulti-modal retrievalVideo